Mid-level Representation for Visual Recognition

Visual Recognition is one of the fundamental challenges in AI, where the goal is to understand the semantics of visual data. Employing mid-level representation, in particular, shifted the paradigm in visual recognition. The mid-level image/video representation involves discovering and training a set of mid-level visual patterns (e.g., parts and attributes) and represent a given image/video utilizing them. The mid-level patterns can be extracted from images and videos using the motion and appearance information of visual phenomenas. This thesis targets employing mid-level representations for different high-level visual recognition tasks, namely (i) image understanding and (ii) video understanding. In the case of image understanding, we focus on object detection/recognition task. We investigate on discovering and learning a set of mid-level patches to be used for representing the images of an object category. We specifically employ the discriminative patches in a subcategory-aware webly-supervised fashion. We, additionally, study the outcomes provided by employing the subcategory-based models for undoing dataset bias. In the case of video understanding, we first study acquiring a mid-level motion-based representation (i.e., tracklet) to capture the commotion of a crowd motion for the task of abnormality detection in crowd. Next, we study exploiting dynamics of a dictionary of appearance-based models (i.e. Poselets) as human motion representation for activity recognition. Our empirical studies show that employing richer mid-level representations can provide significant benefits for visual recognition (both in image and video understanding). Our main contributions are as follows: (a) introducing a method for undoing dataset bias using subcategory-based models, (b) discovering and training webly-supervised subcategory-aware discriminative patches for object recognition, (c) proposing tracklet-based commotion measure for abnormality detection in crowd, (d) introducing Temporal Poselet a descriptor for group activity recognition.